1. Field of the Invention
The present invention relates to a technique for calibrating parameters associated with an imaging apparatus and a display apparatus.
2. Description of the Related Art
Conventionally, a video see-through type head mounted display which fetches a video picture captured at substantially the same position as the pupil position of an observer into a computer or the like and displays that video picture while superimposing a computer graphics (CG) on it has been proposed. Also, a handheld display having the same functions as those of the above display has been proposed.
An example of the head mounted display will be described below using FIGS. 5 to 7.
FIG. 5 is a block diagram showing the hardware arrangement of the conventional video see-through type head mounted display. Note that R and L that follow reference numerals mean components for the right eye and left eye, and form pairs. Therefore, components denoted by the same reference numerals with R and L respectively have the same function, and a description will be given while omitting R and L.
As shown in FIG. 5, the head mounted display comprises a head mounted unit 501 and a control unit 500.
Referring to FIG. 5, reference numeral 110 denotes an imaging element such as a CCD, CMOS area sensor, or the like, which captures a moving image. Reference numeral 111 denotes an analog image processor which includes AGC (automatic gain control), CDS (correlated double sampling), ADC (analog-to-digital converter), and the like, and applies image processing to a video signal from the imaging element 110. Reference numeral 112 denotes a digital image processor which includes γ (gamma) correction, hue correction, edge correction, distortion correction for an imaging optical system, and the like, and applies image processing to a signal which is converted into a digital signal by the analog image processor 111.
Reference numeral 113 denotes a captured video output unit such as an NTSC encoder or the like, which converts a signal that has undergone the image processing by the digital image processor 113 into a predetermined video format, and outputs the converted signal. Reference numeral 120 denotes a display video input unit which includes PLL (phase locked loop), ADC (analog-to-digital converter), and the like, and fetches a video signal output from an external computer. Reference numeral 123 denotes a display video conversion unit which converts the resolution and frame frequency of a video picture based on a video signal input from the display video input unit 120 in correspondence with a compact display element 125, and performs distortion correction for a display optical system. Reference numeral 124 denotes a display element drive unit which includes a controller, driver, and the like for the compact display element 125. Reference numeral 125 denotes a compact display element such as an LCD (liquid crystal display), EL element, or the like.
Reference numeral 101R denotes a right-eye video capturing and display unit; and 101L, a left-eye video capturing and display unit. Reference numeral 130 denotes a controller such as a CPU or the like; 131, a distortion correction table unit for the imaging optical system; and 132, a distortion correction table unit for the display optical system. Reference numeral 502 denotes a cable for connecting the head mounted unit 501 and control unit 500.
FIG. 6 is a view showing the internal arrangement of the conventional video see-through type head mounted display. Note that FIG. 6 does not particularly illustrate a circuit board which mounts the circuit shown in FIG. 5 to allow easy understanding of the overall arrangement, and illustrates only the imaging element 110 and compact display element 125. Referring to FIG. 6, reference numeral 610 denotes a beam of external light as incoming light that comes from an external world. Reference numeral 611 denotes a wedge-shaped prism which bends the optical axis of incoming light in a direction different from the incoming direction to bring the position of the imaging element 110 close to that of an eyeball 622 of the observer as much as possible, and assures a larger distance. Reference numeral 612 denotes an imaging lens which forms an image of the beam 610 of the external light on the imaging element 110. Reference numeral 622 denotes an eyeball of the observer. Reference numeral 621 denotes a beam of display unit video light which emerges from the display element 125 and enters the eyeball 622 of the observer. Reference numeral 620 denotes a sculptured surface prism which enlarges light emerging from the display element 125, and guides it as a beam of light with a certain width to the eyeball 622 of the observer.
FIG. 7 is a view showing the outer appearance when the observer wears the conventional video see-through type head mounted display. Referring to FIG. 7, reference numeral 701 denotes a head of the observer. Reference numerals 101R and 101L denote right and left video capturing and display units; 501, a head mounted unit of the head mounted display; 500, a control unit of the head mounted display; and 502, a cable for connecting the head mounted unit 501 and the control unit 500.
The overall arrangement and the functions of respective units will be explained using these three figures. The observer wears the head mounted unit 501 of the head mounted display on his or her head 701 so that the head mounted unit 501 is located in front of the eyeballs 622, as shown in FIG. 7. The worn head mounted unit 501 includes the right-eye video capturing and display unit 101R for the right eye and the left-eye video capturing and display unit 101L for the left eye, each of which has the arrangement and layout, as shown in FIG. 6. That is, in the imaging system, the beam 610 of the external light enters from the external world, and the optical axis of the incoming light is bent in a direction different from the incoming direction by the wedge-shaped prism 611. In addition, the prism 611 assures a longer distance and the imaging lens 612 forms an image of the incoming light on the imaging element 110, thus bringing the position of the imaging element 110 close to that of the eyeball 622 of the observer as much as possible.
In this manner, a video picture which roughly matches an image actually seen by the observer's eyes can be captured. Also, in the display system, the sculptured surface prism 620 enlarges light emerging from the compact display element 125, and guides it to the eyeball 622 of the observer as the beam of light 621 having a certain width, so that the observer can observe an enlarged virtual image of the compact display element 125. The video picture captured by the imaging element 110 is processed by the arrangement shown in FIG. 5, and is transferred to an external computer. The external computer outputs a video picture obtained by superimposing a CG and the like on the transferred video picture to the head mounted display again, thus displaying that video picture on the display element 125.
That is, the analog image processor 111 applies, to a video picture captured by the imaging element 110, image processing such as AGC (automatic gain control), CDS (correlated double sampling), ADC (analog-to-digital converter), and the like in an analog signal state. The digital image processor 112 applies image processing such as γ (gamma) correction, hue correction, edge correction, and the like to the digital signal that has undergone the image processing by the analog image processor 111. At the same time, this digital signal undergoes image processing for correcting an imaging optical distortion based on data in the distortion correction table unit 131 for the imaging optical system.
The captured video output unit 113 converts the video signal that has undergone the image processing into a predetermined video format by the NTSC encoder or the like, and outputs the converted signal to the external computer. The external computer generates a video picture obtained by superimposing a CG and the like on this captured video picture, and inputs the generated video picture from its standard video output to the display video input unit 120 in a predetermined format such as VGA or the like. The display video input unit 120 generates pixel clocks from a sync signal using the PLL (phase locked loop). The ADC (analog-to-digital converter) converts an analog video signal input from the external computer into a digital video signal, which can be used in the subsequent processing circuits.
The display video conversion unit 123 converts the resolution and frame frequency of the digital video signal into the resolution of the compact display element 125 and the frame frequency suited to display. At the same time, the digital video signal undergoes image processing for correcting a display optical distortion based on data in the distortion correction table 132 for the display optical system. The controller, driver, and the like in the display element drive unit 124 process the digital video signal to a signal suited to drive the compact display element 125, and the compact display element 125 such as an LCD (liquid crystal display), EL element, or the like displays the video picture. Note that correction of a display optical distortion is described in, e.g., Japanese Patent Laid-Open No. 10-327373.
The controller 130 controls all these system units.
Since the head mounted unit 501 is demanded to have a lightweight as much as possible, it includes only the right-eye video capturing and display unit 101R and left-eye video capturing and display unit 101L, and other circuits are incorporated in the control unit 500. The cable 502 connects these two units, and has a length of 5 to 10 m under the assumption that the observer (701) moves around.
Note that the optical distortions differ depending on individuals due to the influences of individual differences, erection tolerances, and the like of optical parts. Therefore, in the distortion correction table unit 131 for the imaging optical system and distortion correction table unit 132 for the display optical system, distortion data are measured for respective individuals and measured data are written.
However, in the conventional video see-through type head mounted display, the head mounted unit and control unit must be indispensably paired. If these units are not paired, when the head mounted unit is exchanged, distortion correction values corresponding to the distortion data of the head mounted unit after exchange must be written in the distortion correction tables by some method.